AD-A251  373 

lllillillili! 


Managemeru  ^ysTems  Laboratories 


1900  Kraft  Drive 
Blacksburg,  Virginia  24060 
(703)231-3501  •  FAX  (703)  231-3538 


Year  2;  First  Quarterly  Report  ONR  Contract  #  N00014-91-J-1500 


Automating  a  Detafled  Cognitive  Task  Analysis  for  Structuring  Curriculum 


Defense  Technical  Information  Center 
Building  5,  Cameron  Station 
Alexandria,  Virginia  22304-6145 


To  whom  it  may  concern: 


d2  6  01  02:0 


Over  the  past  three  months  we  have  made  considerable  progress  in  the 
development  of  our  cognitive  analysis  authoring  system.  A  presentation  and 
demonstration  of  our  developments  was  given  at  ONR  on  May  7th,  1992.  I  feel  our 
efforts  were  well  appreciated  as  per  the  interest  generated  by  the  participants. 


We  are  continuing  to  design  and  implement  refinements  to  the  system  in  order  to 
elicit  greater  detail  especially  with  regard  to  primitive  mental  operators.  Our  initial 
evaluations  have  indicated  users  are  typically  not  cognizant  of  these  mental  operators 
and  consequently  do  not  record  them  when  building  their  models.  We  have  completed 
the  design  for  the  Decide  operator  which  appears  to  be  most  readily  elicited  by  users. 
This  includes  IF-THEN,  IF-THEN-ELSE,  and  GOTO  operators.  We  have  FTT’d  a 
copy  of  Word  Net  and  have  identified  synonyms  for  decide,  retain,  recall,  and  retrieve. 
The  system  will  monitor  the  use  of  these  terms  and  their  synonyms  and  will 
automatically  set  them  to  the  primitive  operator  status.  If  decide,  its  synonyms  or 
statements  containing  IF,  THEN,  or  ELSE  are  detected  by  the  system,  the  user  will  be 
presented  with  a  dialogue  box  explaining  that  a  decision  is  implied  by  the  step 
description.  A  format  for  specifying  conditions  and  operators  to  be  executed,  if  the 
conditions  are  met,  is  presented  to  the  user.  We  have  allowed  for  "anding"  any  number 
of  conditions  in  this  format.  To  create  conditionals  which  require  an  "or"  the  user  is 
instructed  to  create  another  decide  step  immediately  succeeding  the  current  decide  step. 
This  seemed  to  be  the  most  explicit  way  of  handling  decide  steps.  As  for  the  other 
primitive  operators,  no  special  formats  are  required,  they  are  handled  like  any  other 
step.  The  problem  is  going  to  be  eliciting  steps  like  recall,  refrain,  forget,  and  retrieve. 
These  are  working  memory  and  long  term  memory  operators.  Very  few  users,  if  any, 
will  be  familiar  with  these  operators  and  how  they  should  be  used  excepting  for  those 
that  are  intimately  familiar  with  GOMS.  We  are  attempting  to  provide  some  facilitation 
as  Stan  suggested  during  the  training  and  familiarization  sessions  proceeding  each 
subjects  interaction  with  the  tool. 


Our  experimentation  is  designed  to  determine  if  users  can  generate  accurate 
models  of  a  Macintosh  "cut  and  paste"  document,  editing  task  by  comparing  the  models 
generated  by  the  tool  to  a  baseline  model  which  David  Kieras  develo^d.  We  are  also 
going  to  analyze  for  the  consistency  of  the  models  generated  by  test  subjects  among 
themselves  independent  of  the  baseline.  We  may  not  be  getting  completely  accurate 
models  due  to  tnese  mental  operators.  However,  we  may  be  getting  very  consistent 
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models  between  users.  It  may  turn  out  that  some  expert  at  GOMS  will  have  to  edit  a 
resultant  user  model  by  inserting  these  mental  operators  where  appropriate.  David  and 
I  are  also  explaining  the  possibility  of  detecting  overt  primitive  steps  which  would 
regularly  imply  a  primitive  mental  operator  either  precede  or  succeed  the  primitive  step. 
There  may  however,  be  too  many  conditions  which  must  be  tested  in  order  to  make 
such  inferences.  This  gets  us  into  language  dependent  types  of  problems.  Maybe  Word 
Net  can  help? 

We  are  also  changing  our  time  line  in  order  to  take  advantage  of  the  SPAWAR 
project  needs.  As  you  may  recall  we  proposed  to  apply  the  tool  in  the  third  year.  We 
believe  we  are  far  enough  along  to  accelerate  the  application  of  the  tool  for  developing 
and  structuring  curriculum  content  for  SURTASS  console  operator  training.  These 
mental  operators  which  are  giving  us  a  headache  are  needed  for  simulation  models  of 
user  computer  interfaces  but  would  not  typically  be  used  in  instruction.  That  is,  we 
would  not  include  in  an  instructional  setting  statements  such  as  "forget  what  is  in 
working  memory"  or  "recall  what  is  in  working  memory". 

In  parallel  I  will  be  contacting  Sharon  Denny  and  John  Anderson  to  develop 
some  algebra  models  using  the  tool  and  to  get  some  feedback  from  them  as  to  its 
applicability  in  developing  mathematical  models  for  instruction. 

Thanks  for  sending  me  the  information  on  the  Advanced  Instructional  Design 
Advisor,  it  made  for  good  reading.  I  will  be  reporting  on  the  trip  to  Berlin  to  fill  you  in 
on  how  our  research  is  being  received  by  the  European  community. 


Sincerely, 


Kent  E.  Williams 
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